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ON  EMPIRICAL  BAYES  PROCEDURES  FOR  SELECTING  GOOD 
POPULATIONS  IN  POSITIVE  EXPONENTIAL  FAMILY  ^ 


Shanti  S.  Gupta  and  Jianjun  Li 

Department  of  Statistics  Department  of  Statistics 
Purdue  University  Purdue  University 

W.  Lafayette,  IN  47907  W.  Lafayette,  IN  47907 

Abstract: 

The  problem  of  selecting  good  ones  compared  with  a  control  from  k{>  2)  positive  ex¬ 
ponential  family  populations  is  considered  in  this  paper.  A  nonparametric  empirical  Bayes 
approach  is  used  to  construct  the  selection  procedures.  It  has  been  shown  that  the  risks  of 
the  empirical  Bayes  procedures  converge  to  the  (minimum)  Bayes  risk  with  a  rate  of  0(1  fn), 
where  n  is  the  number  of  accumulated  past  observations  at  hand.  Simulations  were  carried 
out  to  study  the  performance  of  the  procedures  for  small  to  moderate  values  of  n.  The 
results  of  this  study  are  provided  in  the  paper. 
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1.  Introduction  and  Formulation. 

In  this  paper,  we  are  interested  in  the  problem  of  simultaneous  inference  and  selection 
from  among  k{>  2)  populations  in  comparison  with  a  standard  or  control.  The  populations 
are  denoted  by  tti,  Trjt.  The  random  variable  Xi  associated  with  TTj  has  the  density 
f{xi\di)  =  c{0i)e-^'^^'h{xi)  with  h{x)  >  0  in  (0,oo),  where  the  unknown  parameter  6i  is  the 
characterization  of  population  tTi. 

Let  Oo  denote  a  standard  or  a  control.  In  practical  situations,  we  desire  to  differentiate 
between  good  and  bad  populations  and  select  good  ones  and  exclude  bad  ones.  Here  a  popu¬ 
lation  TT,-  is  said  to  be  good  if  Oi  >  $o  and  bad  otherwise.  This  type  of  decision  problem  has 
been  considered  by  many  authors.  For  example,  see  early  papers:  Paulson  (1952),  Dunnett 
(1955),  Gupta  and  Sobel  (1958),  Lehmann  (1961),  and  later:  Gupta  and  Hsiao  (1983),  and 
more  recently:  Gupta,  Liang  and  Rau  (1994),  Gupta  and  Liang  (1999),  among  others. 

Let  n  =  {9  =  {9u---,6k}  :  0,  >  0,i  =  1,2, •••,/:}  be  the  parameter  space.  Let  A  = 
{a  =  {oi,  •  •  • , ajk}  :  Oj  =  0  or  1, i  =  1,  •  •  • , fc}  be  the  action  space,  where  Oi  =  1  means  that 
population  TTj  is  selected  as  good,  a,-  =  0  means  population  is  excluded  as  bad. 

The  loss  function  we  use  is 

k 

(1.1)  L{9,d)  =  '^l{ei,ai) 

1=1 

with 

l{di,ai)  =  o, •0,(^0  “  ^i)Il0i<9o]  +  (1  “  ~  ^o)^[9i>eo]- 

We  also  assume  that  9i  is  a  realization  of  a  random  variable  0,,  and  ©i,  •  •  0*  are  inde¬ 

pendently  distributed  with  priors  Gi,  •  •  •,  G*  respectively.  Let  G  =  fliLi  Gi{9i). 

Let  X  =  (Xi,  •  •  • ,  Xjfe)  and  X  be  the  sample  space  of  X.  Here  Xi  may  be  thought  of  as  a 
sufficient  statistic  based  on  several  i.i.d.  samples. 

The  selection  procedure  5  =  ((^i,  •  •  •  ,4),  where  5i{x)  is  the  probability  of  selecting  popu¬ 
lation  TTj  as  good  when  X  =  x  is  observed.  To  ensure  that  the  Bayes  rule  exists,  we  assume 
/o°°  e^dGiie)  <  00  for  z  =  1,  •  •  •,  fc. 

Based  on  previous  assumptions,  a  straightforward  computation  shows  that 

(1.2)  R{Gj)=i^RiiG,Si) 

i=l 

and 

(1.3)  Ri{G,5i)=  f  Si(x)[]lfj{xj)]wi{xi)h{xi)dx +  Ti 
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where 

roo 

fi(Xi)  =  .  ^e)e--‘/‘h{xi)dGM, 

Jo 

w,(x,)  =  re(e^-e)c{e)e-“i>dG,(e). 

Jo 

roo 

T,  =  /  «(9  -  «„)/|,>,„|<iG,(9). 

Here  fi{xi)  is  the  marginal  density  of  X,  and  %  is  independent  of  the  selection  rule  5.  Clearly, 
a  Bayes  selection  procedure  6g  =  {^Gii  •  •  •  i  ^Gk)  is  given  by 


(1.4) 


^Gi  —  { 


1 

0 


if  Wi{xi)  <  0, 
if  Wi{xi)  >  0. 


Let  ai{xi)  =  j^ec{e)e-^^l^dGi{e)  and  i)i{xi)  =  e'^c{e)e-^'l^dGi{e).  Denote  (l>i{xi)  = 

'if)i{xi)/ai{xi).  Then  Sa  can  be  expressed  as 


(1.5) 


SGi  =  { 


1 

0 


if  (j)i{xi)  >  00, 
if  ^i(^t)  ^  00' 


If  Gi  is  unknown,  the  Bayes  rule  cannot  be  applied  and  the  selection  cannot  be  made. 
The  empirical  Bayes  approach  is  a  way  to  help  one  to  make  the  decision  when  past  data 
are  available.  Since  Robbins  (1956,  1964)  introduced  the  empirical  Bayes  approach,  it  has 
become  a  powerful  tool  in  decision-making.  Empirical  Bayes  approach  in  statistical  inferences 
has  been  used  recently  by  Singh  and  Wei  (1992),  van  Houwelingen  and  Stijnen  (1993),  Pensky 
(1998),  Pensky  and  Singh  (1999),  and  Liang  (2000a,  2000b). 

For  each  z  =  1, . . . ,  fc,  let  (X^,  Qij),j  =  1, 2, ...  be  random  vectors  associated  with  popu¬ 
lation  TTj  and  stage  j,  where  X^  is  observable  while  Qij  is  unobservable.  It  is  assumed  that 
Qij  has  a  prior  distribution  G,-,  for  all  j  —  1,2, ,  and  conditioning  on  0^  =  0ij,  Xij  follows 
a  distribution  with  density  f{xij\0ij)  and  (Xy,0ij),z  =  1, . . . ,  A:,  j  =  1,2, . . .  are  mutually 
independent.  At  the  present  stage,  say,  stage  n  +  1,  we  have  observed  X  =  x.  The  past 
accumulated  observations  are  denoted  by  (Xi,  •  •  • ,  X„)  =  Xn,  where  Xj  =  (Xi^-,  •  •  • ,  Xkj)  is 
the  observation  at  stage  j.  Based  on  X„  and  x,  we  wish  to  construct  an  empirical  Bayes  rule 
to  select  all  good  populations  and  to  exclude  all  bad  populations.  Such  an  empirical  Bayes 
rule  can  be  expressed  as 


^n(^)  Xn'j  (^nl (^7  X",.^) ,  *  *  '  ,  dfiki^X,  Xfij ) 

where  5„i(x,X„)  is  the  probability  of  selecting  as  good  if  X„  and  x  are  observed.  Let 
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R{G,  5n)  denote  the  overall  Bayes  risk  of  6n-  Then 

(1.6)  R{G~6n)  =  Y,Ri{G^Sni). 

1=1 

where 

(1.7)  Ri{G,  6ni)  =  f  E[6ni{x,  X)]  ■  [H  '  Wiixi)h{xi)dx  + 

The  regret  Bayes  risk  is  defined  as  r„  =  R{G,5n)  —  R{G,5g),  which  is  used  to  measure 
the  performance  of  empirical  Bayes  rule  Sn-  If  =  o(l),  we  say  that  6n  is  asymptotically 
optimal  (a.o.).  If  =  0{Pn)  for  some  positive  Pn  such  that  limn->oo^n  =  0?  we  say  that 
is  asymptotically  optimal  at  a  rate  of  0{Pn)- 

The  aim  of  this  paper  is  to  construct  an  empirical  Bayes  rule  S„  for  the  selection  problem 
described  above.  For  most  distributions  in  the  family  /(xij^i),  under  the  above  general 
setting  or,  in  some  cases,  with  one  additional  condition  9^dG{9)  <  oo,  we  show  that 
nr„  -4  m,  where  m  is  a  computable  constant  depending  on  G. 

It  should  be  pointed  out  that  Gupta  and  Liang  (1999)  studied  the  selection  problem 
for  gamTna{x\9,  s)  populations,  a  special  case  of  above  problem,  firstly  through  an  empirical 
Bayes  approach.  They  constructed  an  empirical  Bayes  rule  and  established  its  convergence 
rate  0{n~^)  under  some  regularity  conditions.  A  rate  of  0(n~^  log  n)  was  obtained  there 
under  the  condition  that  0^s  are  bounded.  Gupta  and  Liese  (2000)  showed  that  the  limiting 
distribution  of  nRn  is  a  linear  combination  of  independent  random  variables,  where  Rn 
is  the  conditional  regret  of  a  modified  version  of  rule  5*. 

The  remaining  part  of  this  paper  is  organized  as  follows.  In  Section  2,  an  empirical  Bayes 
selection  rule  is  constructed.  The  asymptotic  behavior  of  is  investigated  in  Section  3. 
In  Section  4,  we  provide  a  few  typical  examples  as  applications  of  our  results.  The  proofs  of 
our  results  are  given  in  Section  5. 

2.  Construction  of  Empirical  Bayes  Selection  Procedure  5„. 

The  construction  of  5n  can  be  divided  into  three  steps.  First,  we  construct  an  estimator 
of  Wi{x).  Second,  we  localize  the  Bayes  rule.  And  then  we  complete  the  construction  by 
mimicking  the  Bayes  rule  using  the  estimator  of  w<(x). 

The  construction  of  an  estimator  of  Wi{x)  follows  the  idea  of  Gupta  and  Liang  (1999). 
For  the  loss  function  (1.1),  an  unbiased  and  consistent  estimator  of  Wi{x)  can  be  obtained. 
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For  each  i  =  1,  ■  •  k,  j  =  I,  ■  •  n,  and  a:  >  0,  define 

Qo  H”  ^  T 

(2.1)  Vij(l)  = 

Through  a  standard  calculation,  we  have  E[Vij{x)]  =  Wi{x).  Based  on  this  nice  property,  an 
unbiased  and  consistent  estimator  of  Wi{x)  can  be  constructed  as: 

(2.2)  Wni{x)  =  ^j2^iji^)^ 

^  j=i 


for  each  i  =  I,  ■■■,  k,  and  x  G  (0,  (X)). 

We  call  the  next  step  as  a  localization  of  the  Bayes  test.  Examining  the  Bayes  selection 
rule  6g,  we  see  that  5^  depends  on  x  only  through  Xi.  Also  (f>i{x)  is  increasing  for  i  =  1, 

•  •  •,  k.  If  Xi  is  large  so  that  <j)i{xi)  >  9o,  we  have  =  1;  If  is  small  so  that  (pi{xi)  <  9o, 
we  have  Sa  =  0.  Since  G  is  unknown,  we  do  not  know  at  which  point  we  should  accept  Hq 
or  reject  it.  But,  one  will  be  more  likely  to  take  action  a,-  =  1  if  the  observation  of  Xi  =  Xi 
is  quite  large  and  take  action  Of  =  0  if  it  is  quite  small.  By  knowing  this,  we  want  to  find 
two  numbers  and  L„  such  that  we  select  TTi  as  good  if  we  observe  Xi  >  Ln  and  exclude  it 
as  bad  if  Xj  <  Bn-  Here  both  B„  and  L„  depend  on  n.  This  could  be  understood  as  follows. 
As  n  increases,  we  have  more  information  from  the  accumulated  data,  and  we  should  adapt 
new  Bn  and  so  that  our  decision  can  be  made  more  precisely. 

Certainly,  the  exact  form  of  f{x\e)  and  the  distribution  G  affect  the  choice  of  B„  and  !«• 
Since  we  have  no  knowledge  about  G  except  that  /o°°  edG{e)  <  oo,  we  rely  on  f{x\d)  itself. 

If  \im^ioh{x)  >  0,  let  B„  =  1/L„  and  =  (0ologn/4)  V  10.  If  \im^ioh{x)  =  0,  let 
and  Ln  be  the  two  sequences  of  positive  numbers  such  that  and  if„  -4  oo, 

L„  ^  oo  as  n  -4  oo.  For  example,  L„  =  {9o\ogn/12)  V  10  and  Hn  =  Then 

define  B„  =  [inf{a:  <  1 :  h{x)  <  1/B„}V(1/L„)]  AO.l.  It  follows  that  B„  0  since  -4  oo 
as  n  ^  oo. 

According  to  what  we  mentioned  at  the  beginning  of  this  section,  we  propose  the  following 


empirical  Bayes  procedure:  For  each  i  =  1,  •  •  •,  fc,  and  Xi, 


(2.3) 


1  if 
0  if 


{xi  >  Ln)  or  {Bn  <Xi<Ln  and  Wni{xi)  <  0), 
{xi  <  Bn)  or  {Bn  <Xi<  Ln  and  Wni{xi)  >  0). 


This  empirical  Bayes  procedure  says  that,  at  stage  n  +  1,  if  the  present  observation  Xi  from 
TTt  is  relatively  big  or  small,  a  decision  will  be  made  based  on  Xi  only.  If  it  is  not  too  small  or 
too  big,  we  have  to  resort  to  past  data  information  and  use  1^01(2^))  fhe  estimator  of  Wi{x), 


to  make  the  decision. 
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3.  Asymptotic  Optimality  of 

In  this  section,  the  asymptotic  behavior  of  Sn  is  investigated.  We  derive  the  regret  Bayes 
risk  first.  From  (1.2)  and  (1.3),  the  Bayes  risk  of  Sq  is  R{G,6g)  =  Ei=i  Ri{G,SGi)  with 

roo 

Ri{G,SGi)=  /  6Gi{x)wi(xi)h{xi)dxi+Ti. 

Jo 

From  (1.6)  and  (1-7),  the  Bayes  risk  of  Sn{x)  is  R{G,5n)  =  Ri{G,Sni)  with 

fOO  ^ 

Ri{G,5ni)=  /  E[6ni{x)]wi{xi)h{xi)dxi+Ti. 

Jo 

Thus  the  regret  Bayes  risk  of  5„,  the  difference  between  R{G,Sn)  and  R{G,6g),  is 

k 

(3.1)  fn  ~  ^ 

t=l 


where 


(3.2)  r„i  =  /  PiWniix)  <0)wi{x)I[^.^x)>o]Hx)dx+  f  P{Wni{x)  >  0)'Wi{x)I[y,.(x)<Q]h{x)dx. 
Jb„  JBn 


Under  the  assumption  6^dGi{0)  <  oo,  we  have  \wi{x)\h{x)dx  <  oo  from  the  inequality 

f  \wi{x)\h{x)dx  <  6o  f  ai{x)h{x)dx  +  [  ij)i{x)h{x)dx  <  Oo  [  0dGi{0)  +  (  9^dGi{6). 

Jq  Jo  Jo  Jo  Jo 

Since  Wn{x)  is  a  consistent  estimator  of  Wi{x),  P{Wni{x)  <  0)  — )■  0  if  Wi{x)  >  0,  and 

P{Wni{x)  >  0)  — )•  0  if  Wi{x)  <  0.  Applying  the  dominated  convergence  theorem,  we  have 

r„j  =  o(l).  Thus  we  have  the  following  theorem. 


Theorem  3.1.  Assume  that  9^dGi{9)  <  oo  for  each  i  =  1,2,  Then  as 

defined  by  (2.3),  is  asymptotically  optimal. 


Besides  the  asymptotic  optimality,  the  convergence  rate  of  an  empirical  Bayes  procedure 
is  also  an  important  factor  to  be  considered  when  the  procedure  is  applied.  The  following 
discussion  shows  that  the  procedure  achieves  the  rate  of  convergence  of  order  0{n~^). 

From  now  on,  we  consider  only  those  members  of  the  family  f{x\9)  in  which  limjioo  h{x)  > 
0  and  h{x)  is  bounded  from  below  for  any  inner  closed  subset  of  (0,  oo).  These  members 
belong  to  one  of  the  following  cases: 

Case  1.  lim^too  ^  >  0  and  lim^io  h{x)  >  0. 

Case  2.  limxtoo  ^  >  0  and  limjip  h{x)  =  0. 

Case  3.  lim^-foo  ^  =  0  and  limxio  h{x)  >  0. 
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Case  4.  liin2;-|-oo  ^  —  0  and  lini3;|o  h(^x)  —  0. 

Before  presenting  the  main  results,  wa.  introduce  the  following  definition.  If  Soi  =  1  for 
all  X  e  (0,  oo)  or  Sa  =  0  for  all  x  e  (0,  oo),  we  say  that  Soi  is  degenerate;  otherwise  we  say 
that  Soi  is  non-degenerate. 

If  Sci  is  non-degenerate,  i.e.,  lim^io  <  liniitoo  then  (l)i{x)  is  strictly 

increasing.  Therefore  there  exists  a  point  Cj  G  (0,  oo)  such  that  (f)i{ci)  =  ^O)  for 

X  >  Ci  and  <  9o  for  x  <  Ci. 


Theorem  3.2.  Assume  that  e^dGi{e)  <oo  fori  =  l,  ■■■,  k.  In  Case  3  and  Case  4, 
we  also  assume  that  6'^c{6)dCi{9)  <  oo  for  i  =  1,  •  •  •,  k.  Then 

k 

(3.3)  limnr„  =  Vmi, 

^  n^oo 

t=l 

where 


(3.4) 


mi  = 


if  Sci  is  degenerate, 
if  5g%  is  non- degenerate. 


and  I^ar([Vii(ci)])  is  the  variance  ofVnici),  w[{ci)  is  the  derivative  of  Wi{x)  at  Ci,  w[{ci)  ^  0 


if  ^Gi  ‘Is  non-degenerate. 


Proof.  The  proof  is  given  in  Section  5. 

In  Case  3  and  Case  4,  the  assumptions  f^  9^dCi{9)  <  oo  and  ff°  9‘^c{9)dCi{9)  <  oo  can 
be  simplified  into  f^  9^ dCi{9)  <  oo.  So  we  have  the  following  corollary. 

Corollary  3.3.  In  Case  3  and  Case  4,  if  /o°°  9^dCi{9)  <  oo  for  i  =  I,  •  •  •,  k,  then  (3.3) 
as  well  as  (3.4)  holds. 

Proof.  Note  that  9c{9)  =  9[f^  exp{-x/9)h{x)dx]~^  and  for  9  >  1, 

roo  roo  r2 

(3.5)  e“®/®/i(x)dx  =  /  e~^h{ye)dy>e~'^  h{ye)dy  >  e~^[rmnh{t)]. 

Jo  Jo  Jl  - 

It  follows  that  9c{9)  is  bounded  for  9  >  1.  Thus  from  9^dCi{9)  <  oo  we  have  both 

f^9‘^dCi{9)  <  oo  and  9‘^c{9)dCi{9)  <  oo.  Then  Corollary  3.3  follows  Theorem  3.2. 


Prom  Theorem  3.2,  one  sees  a  rate  of  order  0{n  ^)  is  obtained  under  a  (quite)  weak 
condition.  We  only  require  f^  9‘^dC{9)  <  oo  in  Case  1  and  Case  2.  The  assumption 
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/o°°  d^dG{6)  <  oo  guarantees  the  existence  of  the  Bayes  rule.  This  assumption  is  natural  and 
not  very  stringent.  In  Case  3  and  Case  4,  A^e  require  one  moment  condition,  /o°°  9^dG{9)  <  oo. 


The  applications  of  our  results  to  a  few  typical  distributions  are  presented  in  the  following 
section.  It  includes  the  construction  of  6n  and  the  statement  of  convergence  rate  for  each 
distribution  there. 


4.  Examples  and  Results. 


We  select  a  few  distributions  as  examples. 


Example  4.1  {exp  {9)- family) .  Consider  the  exponential  populations  having  density 


(4.1) 


Xi  >  0,  9i>0,  i  = 


Here  h{x)  =  1.  This  family  belongs  to  Case  3.  Take  Bn  =  l/I'm  Ln  =  (^o  logn/4)  V  10  and 
construct  as 

1  if  (xi  >  Ln)  or  {Bn  <Xi<Ln  and  Wni{xi)  <  0), 

0  if  {xi  <  Bn)  or  {Bn  <  Xi  <  Ln  and  W„i(xi)  >  0). 

Then  applying  Corollary  3.3,  we  have  the  following  result. 


(4.2) 


Sni{Xi)  =  { 


Result  4.1.  IfXi  has  density  f{xi\9i)  given  in  (4-1)  and  /o°°  9^dGi{9)  <  oo  for  all  i  —  1, 
■■■,  k,  then  5n,  as  constructed  in  (4.2),  has  a  rate  of  convergence  of  order  0{n~^). 

Example  4.2  {Gamma  {9,  s)- family  with  known  s  >  1  ).  Consider  the  gamma  popula¬ 
tions  having  density 


(4.3) 


/W«i)  =  *i>o.  «i>o.  •  = 


Here  h{x)  =  x*“^.  This  family  belongs  to  Case  2.  Let  L„  =  {9ologn/12)  V  10  and  Bn 
|^-i/[6(s-i)]  V  L~^]  A  0.1.  Construct  Sn  as; 

1  if  (Xj  >  Ln)  or  {Bn  Xi  Ln  and  Wni{Xi)  ^  0), 

0  if  (Xj  <  Bn)  or  {Bn  <  Xi  <  Ln  and  Wni{xi)  >  0). 

Then  applying  Theorem  3.2,  we  have  the  following  result. 


(4.4) 
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Result  4.2.  If  Xi  has  density  f{xi\6i)  given  in  (4-3)  and  f^  0^dGi{9)  <  oo  for  alii  —  1, 
k,  then  6n,  as  constructed  in  (4-4)i  hQ.s  a  rate  of  convergence  of  order  0{n  ^). 


Example  4.3  {A  population  having  the  density  with  infinite  many  discontinuities  ).  Con¬ 
sider  the  exponential  populations  having  density 


(4.5)  /(Xil^i)  =  +  >  0,  i  =  l,---,k. 

l-O 

Here  h{x)  =  Ez=o(^  +  l)/[z<x<z+i]-  This  family  belongs  to  Case  1.  Take  =  1/Ln, 
=  (^0  logn/4)  V  10  and  construct  5n  as 


(4.6) 


dni(^i) 


1  if 
0  if 


(xi  >  Ln)  or  {Bn  <Xi<Ln  and  Wni{xi)  <  0), 
(xi  <  Bn)  or  (Bn  <  Xi  <  Ln  and  W„i(xi)  >  0). 


Then  applying  Theorem  3.2,  we  have  the  following  result. 


Result  4.3.  IfXi  has  density  f{xi\0i)  given  in  (4-5)  and  f^  6'^dGi{9)  <  oo  for  all  i  — 
■  ■  k,  then  as  constructed  in  (4-6),  has  a  rate  of  convergence  of  order  0(n“^). 


Remark.  Gupta  and  Liang  (1999)  considered  the  same  selection  problem  for  the  gamma 
population  (4.3).  In  that  paper,  an  empirical  Bayes  rule  was  constructed  as 


f  1  iflT„i(xi)<0, 
jo  ifW„z(xi)>0. 


The  convergence  rate  of  =  (<^;;i,  •  •  • ,  Kk)  is  affected  by  the  tail  probability  of  the  underlying 
distributions.  In  our  paper,  we  split  the  interval  (0,  oo)  into  three  parts  (0,  Bn),  {Bn,  L„]  and 
(L„,oo)  by  localizing  the  Bayes  test.  Then  we  construct  the  empirical  Bayes  rule  as  (4.4). 
So  the  influence  of  the  tail  probability  of  the  underlying  distributions  is  controlled  and  a 
rate  of  0{n~)  is  obtained  under  quite  weak  conditions  as  shown  in  Result  4.2. 


5.  Proof  of  Theorem  3.2. 

The  main  idea  of  the  proof  is  to  use  a  classic  result  about  the  non-uniform  estimation 
of  the  difference  between  the  normal  distribution  and  the  distribution  of  the  sum  of  i.i.d. 
random  variables.  We  shall  prove  it  in  the  following  two  subsections  according  to  whether 
all  Sci  axe  non-degenerate  or  not. 
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5.1.  All  5ai  are  non-degenerate.  We  shall  prove  Theorem  3.2  assuming  that  all 
Sci  are  non-degenerate  in  this  subsection*  Then  there  exists  a  point  q  G  (0,  oo)  such  that 
Mci)  =  Oo,  (j)i{x)  >  $0  for  X  >  Ci  and  <^i{x)  <  Oq  for  x  <  q.  Since  we  consider  the  asymptotic 
behavior  of  6n,  we  assume  Cj  €  {Bn,Ln)  for  all  n  without  loss  of  generality. 

Lemma  5.1.  For  each  k,  u;.(ci)  <  0  and  further  there  is  a  neighborhood  of 

Ci,  denoted  by  7V(cj,e,),  such  that  N{ci,ei)  C  {Bi,Li)  and 

(5.1)  A,  =  >  0. 

Denote  ca  =  Ci- Ci,  Ci2  =  Ci  +  Ci.  Then  for  all  x  G  [S„,  ax]  U  [a2,  Ln],  there  exists  an  Mi  >  0 
such  that 

(5.2)  |rr’i(a:)|  >  Mie-^^'^°. 


Proof.  For  x  >  0,  the  derivative  of  Wi{x)  exists  and  can  be  expressed  as 
w[{x)  =  -9o  e-^^^c{6)dGi{e)  +  ee-^^^c{e)dGi{9). 


Prom  Jensen’s  inequality,  we  see  that  for  x  >  0 

J^e--l‘c{B)dG,{9)  SS°ee-‘/»c(e)dGm  ' 
Plugging  a  for  x  in  the  above  inequality,  we  have 

Cee-»i^c{e)dGde) 

IS°e-^l>c(9)dG,(e)  ”■ 


This  implies  that  w]{a)  <0. 

Note  that  w'i{x)  is  continuous  in  (0,  oo).  So  an  N{a,  a)  can  be  found  such  that  N{a,  a)  C 
{Bi,Li)  and 

A<=  min  |u;-(a;)l  >  0- 

xeN{Ci,ei) 

Then  (5.1)  is  proved.  On  the  other  hand,  rewrite  Wi{x)  as  Wi{x)  =  Q!i(x)[0o  -  <l>i{x)]-  Noting 
that  (/>i(x)  is  strictly  increasing  in  x  and  (f>i{a)  =  9q,  then  for  x  G  [Bn,  ai]  U  [ci2,L„], 


1^0  -  (f>i{x)\  >  i9o  -  (f>i{ai))  A  (0i(Ci2)  -  Oo). 


For  X  <  Ln, 

/'OO  poo 

cciix)  >  I  9c{9)e-^l^dGi{9)  >  /  9c{9)dGi{9). 

JBq  •'Oo 
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Thus 

poo 

|wj(x)|  >  [(00  -  </'i(cii))  A  {^i{ci2)  -  6c{6)dGi{6). 

J  UQ 

This  completes  the  proof  of  Lemma  5.1. 


Next  lemma  deals  with  the  bounds  of  the  moments  of  Wni{x). 

In  Case  1  and  Case  3,  mino<i<oo  ^(2^)  >  0-  Let  5„  =  l/[mino<x<oo In  Case  2  and 
Case  4,  Let  5„  =  i/„  V  [1/ mini<x<oo /i(a;)].  Then  h{x)  >  for  x  >  in  all  four  cases. 
Recall  Ln  =  Bq  logn/4  in  Case  1  ans  Case  3  and  =  nL'*  in  Case  2  and  Case  4.  Then 

we  have  ~ 

In  Case  3  and  Case  4,  we  know  9'^c{9)dGi{0)  <  00  and  let  Ci  =  9^c{9)dGi{9). 

Without  loss  of  generality,  we  assume  h{x)  >  x  for  x  >  1  in  Case  1  and  Case  2. 


Lemma  5.2.  Let  <7f(x)  =  E[{Vij{x)  -  Wi{x))^]  and  7i(x)  =  E[\Vij{x)  -  Wi(x)n.  Then 
for  X  £ 


(5.3) 


[25,1(00  +  1)  +  1]^ 

5n[(0O  +  l)^<^i(^)  +  2(00  +  l)C'i] 


for  Case  1  and  Case  2  , 
for  Case  3  and  Case  4, 


and 


I  4[25„(0o  +  1)  +  1]^  +  4|wi(x)|^ 

(5.4)  7i(x)  <  < 

[  lesim  +  6)ai(^)  +  ec,]  +  Mwi{x)  p 

For  X  £  [cii,  Ci2],  there  exist  a  constants  Ci^  >  0  such  that 
(5.5)  7i  <  Ciy. 


for  Case  1  and  Case  2  , 
for  Case  3  and  Case  4  • 


For  X  £  [Bn,  Oil]  U  [Ci2,  Ln]  and  large  n  , 

(5.6)  n^^®|u;i(x)|/|(Ti(x)|  >  1. 


Proof.  Consider  x  G  [B„,L„].  Note  that  h{x)  >  S-^.  In  Case  1  and  Case  2,  if  x  >  1, 
h{x)  >  X,  and 

|V«(I)|  <  /|x,>.i»o/ft(X,)  +  -  x)/h{Xj)  <  «„S„  +  1. 

If  Bn  <  X  <  1,  it  can  be  shown  that  |Vij(x)|  <  25„(0o  +  1)  +  1.  Thus 

aUx)  <  E[\Vij{x)\^]  <  [25„(0o  +  1)  +  I]"*- 
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For  7i(2;),  using  \a  +  bf  <  4|a|^  +  416|^,  we  have 

'Yiix)  <  AE[\Vij{x)\^]+A\wi{x)\^  <  4[2Sn{eo  +  1)  +  if  +  4|u;i(a;)p. 

Then  (5.3)  and  (5.4)  are  proved  for  Case  1  and  Case  2.  In  Case  3  and  Case  4,  a  simple 
calculation  shows  that 

af{x)  <  Sn[9lai{x)  +  20oMx)  +  2 1"  e^cie)e-^^^dGi{e)]. 

By  breaking  the  interval  (0,oo)  into  (0,1)  and  [l,co),  we  have  ipiix)  <  Ci  +0:^(2:)  and 
l^e^c{e)e-^l^dGi{9)<Ci^ai{x).  Thus 

<  5'„[(0o  +  l)^oti{x)  +  2(^0  +  l)C'i]- 


Similarly, 

7i(rE)  <  165^[(0o  +  6)Q:i(2;)  +  GCj]  +  4|u;i(a:)f . 

Now  consider  x  G  [cii,  Ci2].  It  is  easy  to  see  that 

I  4(i5K^^^l*  +  4maxj„<;,<.,,|wi(i)|^  =  Ci,  in  Case  1  and  Case  2 

-ftW  S  1  +  4max.„<.<.,,  |wi(j:)p  s  C,,  in  Case  3  and  Case  4 

Then  (5.5)  holds.  Next  we  prove  (5.6).  From  (5.2),  |ti;t(rc)|  >  for  x  G  [Bn,Cfi]  U 

[ci2,  Ln].  In  Case  1  and  Case  2, 


^Wi{x)  I  ^  MjC 


ai{x)  ~  2Sni9o  +  1)  +  1 

In  Ceise  3  and  Case  4 
.Wi{x) 


,5-lg-W0o^„-l/4_ 


> 


1^0  -  (l>i{^)\ 


'  cri(i) '  ^  sy2[(«„  +  ly/aix)  +  2(d.  +  l)C,/[a,(s;)|=l‘/2 
It  is  easy  to  see  that  |0o  -  M^)\  >  min{|0o  - <f>i{cii)l  \9o  - <Ai(ci2)l}-  We  know  from  the  proof 
of  Lemma  5.1  that  at(^)  >  9c{9)dGi{9).  Then 

+  l?Mx)  +  2(00  +  m/[a,{x)n^^  ^ 


Thus  \wi{x) ! Oi{x)  \  =  0(5"^/^e“^’*/®®)  =  0(5y^n“^/^).  This  completes  the  proof  of  Lemma 
5.2. 


Note  that  Vij{x)  are  i.i.d  random  variables  for  fixed  x.  For  large  n,  the  central  limit 
theorem  tells  us  that  Ej=i[^ii(3;)  -  Wi{x)]/[ai{x)^/^  is  close  to  N{0, 1)  in  distribution.  Fur¬ 
thermore,  we  have  the  following  non-uniform  estimation  of  the  difference  between  the  normal 
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distribution  and  the  distribution  of  the  sum  of  i.i.d  random  variables.  This  result  can  be 
found  in  Petrov  (1995,  P168).  * 

Fact.  Let  Xi,  X2,  •  ,Xn  be  i.i.d  random  variables,  EXi  =  0,  EX^  =  >  0,  E\Xi\^  < 


00.  Then  for  all  x 


|F„(a:)  -  $(a:)l  <  A- 


Ko.o)  1^1)3- 

Here  $(a:)  is  the  c.d.f.  ofN{0, 1),  F„{x)  and  p  are  given  by 

1  A  X  E\Xi\^ 


Now,  we  are  ready  to  prove  our  main  result. 

Proof  of  Theorem  3.2.  Rewrite  P{Wni{x)  <  0)  as 


/n(Ti(x)  j=i 


Then  applying  (5.8),  we  have 


v/n|w;i(x)L  A'yAx) 

P{Wni{x)  <  0)  <  ^  +  r(  i  \1  r\  t  1113 • 

V  mv  ;  7-  V  y/n{ai{x)  +  y/n\wi{x)\p 


Similarly, 


^/n\wi{x)\. 


Ajiix) 


P{Wni{x)  >  0)  <  1  -  $(^  +  n  (  \  L  r\  (  Mil- 

^  ^  ^  <7i(x)  y/n{ai{x)  +  y/n\wi{x)\p 

Plugging  above  two  inequalities  in  (3.2),  we  obtain 


fCi  ./nwjjx)  _ Anfi{X) 

‘  -  JbJ-  ^  (Ti{x)  ^  y/n{(Ti{x)  +  v^l 


Ajiix) 


+ni_$(x^)+ 

Jci  ai{x)  ^/n[ai 


A'yijx) _ 

(x)  +  v^K(a;)|) 


-]wi{x)h{x)dx 


-]\wi{x)\h{x)dx 


=  I +  11. 


From  (5.3),  (5.4),  (5.5)  and  (5.6),  we  see  that  «;i(a;),  afix)  and  jiix)  have  different  behavior 
for  different  x.  So  we  decompose  I  into  four  parts. 

/,.  ^\  r  pf./  \... J.V.  f  <T./_  ^yj^(^x)h{x)dx 


(5.9)  7  =  r^-^^)Mx)hix)dx+r^-. 

^  JBn  cri{x)  Jen 


(Xi{x) 


fcn  Aji{x)wi{x)h{x)  n  A'yi{x)Wi{x)H{x) 

^  /b„  x/n((7i(a;)  +  v^liyi(a;)|)3  ^  Jen  \/n{ai{x)  +  ^/n\wi{x)\y 

=  7i  +  72  +  7$  +  li- 


f^i  Aji{x)wi{x)h{x) 
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Consider  I\  first.  According  to  (5.6),  as  n  is  large,  Wi{x)/ai{x)  >  n  for  x  €  [Bn-,Cii],  It 
follows  that  y/nwi{x)/ai{x)  >  Then^pplying  it  to  h,  we  have 


(5.10) 


h  <  Wi{x)h{x)dx  =  o{n  ^). 


For  I2,  letting  hi  and  Ci  be  the  maximum  values  of  h{x)  and  (Xi(x)  on  [cii,Ci2\.  Thus 

Jcii  O’i 

Using  (5.1)  and  letting  y  =  ^Jnwi{x)lau 

rCi  CQO 

/  ^{-^/nwi{x) / di)wi{x)dx  <  — ^  /  ^{-y)ydy 

Jcii  *^0 

Then 
(5.11) 


4A,n' 


,.  r  ^  hiaj 

llinSUpn/2  <  -77“* 

n->00  4:A.i 


Next  we  consider  h.  From  (5.2),  lu)i(rc)|  >  for  x  €  [5n,Cii]-  In  Case  1  and  Case 

2,  applying  (5.4),  we  have 

f'n  4|25„(^0  +  1)  +  1]^  +  4|Wi(x)P 
/a,  •Jn(cri(x)  +  0i|mi(x)|)* 

4l2S„(eo  +  1)  +  IP  /■“ 


(5.12) 


/3  < 

< 


-Wi{x)h{x)dx 
4 


o{n-^). 


rCii  4 

-  /  H — -  /  t/;i(x)/i(x)dx 

j  Tt  ''Bn 


In  Case  3  and  Case  4,  using  (5.4)  again, 

165^[(gg  +  6)Qi{x)  +  6Ci]  +  4|u;i(a:)|^ 
y/n{ai{x)  +  >/n|u;i(a;)|)3 

165'^(^o  +  6)  ,  96S'^C'i 


(5.13) 


<  r 

JBn 


< 


Wi{x)h{x)dx 

Is!  Is!  + 


fCil 

JBn 


=  o{n  ^). 


For  a:  G  [ci,Cii],  7i(a:)  <  Qr-  Let  be  the  minimum  value  of  ai{x)  on  [cji.Cja].  It  is  easy 
to  see  that  cri(x)  >  0  for  each  x  G  [cti,Ci2]  C  [Bi,Li].  Noting  that  cri{x)  is  a  continuous 
function  of  a;,  then  >  0.  It  follows  that 


(5.14)  U  <  ACi^hi 


r<^i  Wi{x)dx  ^  AUj^hj  p 

Jen  \/n(^  + -s/n|iWt(a;)l)^  “  Jo 

Combining  (5.9)  to  (5.14),  we  have  limsup„^^n/  <  Note  that  I  is  independent  of  Ci, 
hi  -4  h{ci),  Oi  -)•  Uar([Uii(Q)])  and  Ai  -)■  |iOi(ci)|  as  Cj  ->  0.  Then 


ACi^hi  /•v^“'(c0 


r-^z'>v  = 

(cLi  +  y)3  n 


limsupn/  < 

n-¥oo 


h{ci)Var{[Vii{ci)])  ^  rr^ 

4|Wi(ci)|  2 
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Similarly  limsup„_^t„n/ J  <  mi/2.  Therefore 

limsupnr„i  <  rrii. 

n— J'CX) 


Note  that 


r.  _ 

-  Jb}  ^  Bi(x)  >  yS( 


{ai{x)  +  y/n\wi{x)\y 

+  /^"[l  - 

J Cx  ^i(^) 


]wi{x)h{x)dx 

:]\wi{x)\h{x)dx. 


,{x)  '  v^(ai(a;)  +  0^|«;i(a:)|)3^ 

We  have  proved  that  /t  Sb"  = 

o(n-‘)  and  /4il  -  =  “('>"')■  Then 


lim  inf  uvni  >  lim  inf  n 

n—^oo  n^oo 


y/nwi{x). 


)wi{x)h{x)dx  +  /  '"[1  -  )wi{x)h{x)dx]. 

J  a  CTi  { ^ ) 


ai{x) 


Using  the  same  idea  applied  to  /2,  it  is  easy  to  prove  the  left-hand-side  of  above  inequality 
is  not  smaller  than  rrii.  So  the  proof  of  (3.3)  and  (3.4)  is  complete. 


5.2.  Some  components  of  So  are  degenerate.  We  shall  prove  Theorem  3.2  assuming 
that  some  of  components  of  6g  are  degenerate. 

For  simplicity,  we  assume  that  only  Sqi  is  degenerate  without  loss  of  generality.  We  need 
to  show  Tni  =  o(n"'^).  If  P{0i  =  9o)  =  1,  wi(x)  =  0  and  r„i  =  0.  Assume  P(9i  =  9o)  <  1  in 
the  following.  Prom  the  proof  in  Subsection  5.1,  we  only  need  to  prove  that  (5.2)-(5.4)  and 
(5.6)  hold  for  all  x  €  [Bn,  T„].  Notice  that  (5.3)  and  (5.4)  do  not  depend  on  the  assumption  of 
non-degeneracy  of  Sqi.  Then  we  only  need  to  show  (5.2)  and  (5.6).  If  Gi  is  degenerate,  then 
(5.2)  and  (5.6)  are  obvious.  Next  we  assume  that  Gi  is  non-degenerate  and  limj;4.o  </>i(x)  >  Oq 
or  limxtoo  0i(2;)  <  ^0-  Denote  <^i(0-f)  =  limj4.o<^i(x)  and  ^i(oo-)  =  lim^too  <?!»i(a;)-  We  shall 
show  (5.2)  first. 

If  <^i(0+)  >  9o  and  Gi(9o-)  =  0,  for  x  G  [5„,  L„],  |tyi(x)|  >  f^(9-0o)^c(9)dGi(0) 

and  ax(x)  >  0c(9)dGi(9).  Then  (5.2)  holds. 

If  <^i(0-l-)  >  00  and  Gi(0o—)  >  0,  then  q:i(0)  <  oo  and  ^i(O)  <  oo.  The  reason  is  in  the 
following.  Since  Gi(0o—)  >  0,  we  can  find  e  >  0  such  that  (?i(0o  —  e)  >  0.  From  (3.3),  we 
know  0c(0)  is  bounded  on  [0o-e,  oo).  Then  J^Jc{0)dGi{0)  <  oo.  Therefore,  if  ai(0)  =  oo, 
then  0c{0)e~^^^dGi{0)  oo  as  x  ->  0.  And  since 

{0o-e)I^°-^0c{0)e--/^dGi{0)+J,^_jMO)e-^^^ 

M^)  -  j^o-^ec{0)e-^/«dGi{0)  -h  f^_Jc{0)e-/^dGi{0) 
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we  have  0i(O+)  ^  Oq  —  This  contradicts  (^i(0+)  ^  ^o-  Thus  Q!i(0)  <  oo  and  '0i(O)  <  oo. 
Then  we  have  t/;[(0+)  <  0.  Let  Cn  >  0  such  that  —w[{x)  >  di  >  0  for  x  G  (0,  Cn).  As  n  is 
large,  <  Cn-  Then  for  x  e  [Bn.cu],  x  >  1/L„  and  |w(a:)|  =  l'u;(a;)  -  w{0)\  >  a:|«;i(x*)|  > 
di/L„,  where  x*  G  (0,x).  It  is  easy  to  see  that  (5.2)  is  true  for  x  G  [cii,L„].  Then  (5.2) 
holds  for  all  x  G  [Bn,  T„]  in  this  case. 

If  01  (oo-)  <  6o,  we  must  have  Gi{Oo)  =  1.  The  reason  is  in  the  following.  If  Gi{0o)  <  1, 
let  e  >  0  satisfy  G{9q  +  e)  /  1.  Then 

g^c(g)e-dg'‘-(g°+d-hrfGi(^)  -h  {9o  +  e) 

^  6'c((9)e-d«->-(9o+£)-‘ldGi(^) 

Therefore  limitoo  0i(a;)  >  9o  +  ^-  This  contradicts  0i(oo-)  <  ^o-  Since  Gi{Oo)  =  1, 
u^i(3;)  =  /o^'’  ^(^0  -  9)c{e)e-^^UGi{0)  >  -  9)c{9)dGi{9)  for  a:  G  [Bn,  T„]. 

Next  we  shall  prove  (5.6).  It  is  obvious  for  Case  1  and  Case  2  from  (5.2)  and  (5.3).  We 
only  prove  (5.6)  for  Case  3  and  Case  4. 

If  01  (0+)  >  ^0  and  Gi(^o-)  =  0,  ai(a;)  <  9c{9)dGii6)  <  oo  for  all  x  >  0.  Then  (5.6) 

follows  from  (5.2)  and  (5.3). 

If  01  (0+)  >  ^0  and  Gi(0o-)  >  0,  ai(0)  <  oo  from  previous  result.  Then  (5.6)  follows 
from  (5.2)  and  (5.3). 

If  0i(oo-)  <  9o,  Gi{9o)  -  1  and  0i(x)  <  9o  for  all  x  >  0.  We  know  <  1  for  large 
n.  Then  for  x  G  [l,Tn],  ai(a:)  is  bounded.  Then  (5.6)  follows  from  (5.2)  and  (5.3).  For 
X  G  [5„,  1],  a;i(x)  and  9o  -  0i(x)  are  bounded  from  below.  Then  (5.6)  follows  from  (5.7). 

Now  the  proof  of  Theorem  3.2  is  complete. 


6.  Simulation  Study. 


A  simulation  study  was  carried  out  to  investigate  the  performance  of  the  proposed  em¬ 
pirical  Bayes  selection  procedure  (2.3)  for  small  to  moderate  values  of  n. 

We  consider  exp(0)  family  in  this  simulation  study,  i.e., 


TTf  ~ 


This  is  also  Example  4.1.  We  consider  the  case  in  which  /c  =  3.  That  is,  there  are  3 
exponential  populations  (treatments)  tti,  7r2  and  tts.  We  shall  make  a  selection  using  the 
procedure  (4.2).  Assume  that  the  prior  distributions  Gi,  G^  and  Gz  are  i.i.d.  having  a 
density 


9i{^)  = 


1 

(s-2)!0"^ 


9>0, 
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where  5  >  4  so  that  je^gi{6)d6  <  00  and  the  requirement  of  Corollary  3.3  is  satisfied.  It  is 
ecisy  to  compute  that  * 

s  —  2, 

and  the  marginal  density  of  X 

f.(x)  =  ,  X  >  0 

’  (x  +  1)* 

for  z  =  1, 2, 3.  As  (s  -  2)9q  >  1,  do  is  nondegenerate  and  Cj  =  (s  -  2)6q  -  1  G  (0, 00).  Then 
it  can  be  computed  that 

2(,-3) 

According  Corollary  3.3,  the  regret  of  the  empirical  Bayes  selection  (4.2)  is  close  to  ^  as  n 
is  large.  Note  that 

Wi{x)  _  [^o(s  -  2)  -  (x  +  l)](x  +  1) 

/i(x)  (s  -  l)(s  -  2) 

Following  (3.1)  and  (3.2),  the  regret  of  the  empirical  Bayes  selection  rule  can  be  expressed 
as 

[^o(s-2)-(X,  +  l)](X,  +  l)  1 

Xu[l[Wni{Xi)<0,Xi<(s-2)6o-l,Br.<Xi<Ln]  (5  _  _  2) 

where  the  expectation  is  taken  over  the  probability  generated  by  {Xi,X2,  •  •  • ,  Xn,  X).  Denote 
D  as 


„  A.  [9o(s-2)-(A:,  +  l)l(A-i  +  l), 

i=l 


(»  -  1)(«  -  2) 


The  scheme  of  the  simulation  is  described  as  follows: 

(1)  For  each  i  and  n,  generate  independent  random  variables  as  follows: 


for  i  =  l,...,n, 

<  (a)  first  generate  Qij  from  an  inverse  gamma  distribution  with  density  9i{0), 

(b)  then  generate  Xij  from  an  exponential  exp{9ij)  distribution. 

Likewise,  generate  0^  from  an  inverse  gamma  distribution  with  density  gi{9),  Xi  from 
exp  (9i). 
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Table  1 

Performance  of  the  selection  rule 
when  s  =  6  and  Oq  =  0.6 


n 

Dn 

5£(T)„) 

m 

n 

10 

0.13577026 

0.003197456 

0.16000000 

20 

0.06920616 

0.003120035 

0.08000000 

30 

0.04476671 

0.003669832 

0.05333333 

40 

0.03566289 

0.002416336 

0.04000000 

50 

0.02786863 

0.003738685 

0.03200000 

60 

0.02122455 

0.003135153 

0.02666667 

70 

0.01999013 

0.003188093 

0.02285714 

80 

0.01898793 

0.002517613 

0.02000000 

90 

0.01945126 

0.002356803 

0.01777777 

100 

0.01715247 

0.002609822 

0.01600000 

150 

0.01190080 

0.002147165 

0.01066667 

200 

0.01011384 

0.002052432 

0.00800000 

Likewise,  generate  ©j  from  an  inverse  gamma  distribution  with  density  gi{6),  Xi  from 
exp  {9i). 


(2)  Based  on  the  past  observations  {XjJ  =  and  the  present  observations 

X  =  (Xi, . . . ,  Xk),  we  compute  D. 


(3)  Repeat  steps  (1)  and  (2)  10000  times.  The  average  of  the  D's  from  the  10000  repetitions, 
denoted  by  D„,  is  used  as  an  estimator  of  the  difference  r„.  The  standard  error,  denoted  by 
SE{Dn),  is  also  computed. 

Table  1  gives  the  results  of  this  simulation  study  on  the  performance  of  the  proposed 
empirical  selection  procedure.  For  a  specific  n,  three  columns  give  Z)„,  SE{Dn)  and  In 
this  case  we  choose  s  =  6  and  =  0.4. 

Figure  1  gives  the  plots  of  (n,  £>„)  and  (n,  ^)-  The  dotted  line  gives  the  values  of  the 
solid  line  denotes  the  values  of  D„. 
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Figure  1:  Graph  for  Table  1. 
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